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@) Coding and decoding methods and systems. 

(57) Input data is efficiently variable length coded 
by using a variable length coding table selected 
from among a plurality of variable length coding 
tables (23C, 23D) in accordance with a coding 
efficiency. It is thus possible further to improve 
variable length coding efficiency as compared 
with a case using only one variable length 
coding table. As a result, when generating infor- 
mation content equal to that generated by using 
only one variable length coding table, it is 
possible to process quantized data with a smal- 
ler quantization size and further to improve the 
quality of information transmitted as coded 
data. 
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(S) Coding and decoding methods and systems. 

@ Input data is efficiently variable length coded 
by using a variable length coding table selected 
from among a plurality of variable length coding 
tables (23C, 23D) in accordance with a coding 
efficiency. It is thus possible further to improve 
variable length coding efficiency as compared 
with a case using only one variable length 
coding table. As a result, when generating infor- 
mation content equal to that generated by using 
only one variable length coding table, it is 
possible to process quantized data with a smal- 
ler quantization size and further to improve the 
quality of information transmitted as coded 
data. 
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This invention relates to coding and decoding 
methods and systems, and is applicable, for example, 
to variable length coding and decoding methods and 
systems for orthogonally transformed picture signals. 

Video signal transmission systems for transmit- 
ting video signals in the form of motion pictures to re- 
mote places, such as a video conference system and 
videophone, have been proposed. In the video signal 
transmission systems, video signals are coded by us- 
ing line correlation and inter frame correlation of a vid- 
eo signal in order efficiently to use a transmission 
path, thereby improving the transmission efficiency 
of significant information. 

For example, intra frame coding uses the line cor- 
relation of video signals. Therefore, as shown in Figs. 
1A and 1B, when transmitting pictures PC1, PC2, 
PC3, ... at time points t = t 1f t 2 , t 3 constituting mo- 
tion picture, the picture data to be transmitted is one- 
dimensionally coded in the same scanning line before 
transmitting the picture data. 

Inter frame coding improves compressibility by 
using the inter frame correlation of video signals to 
obtain picture data PC12, PC23, ... having differenc- 
es in pixel data between successively adjacent pic- 
tures PC1 and PC2, PC2 and PC3 

Thereby, the video signal transmission system is 
constituted to code the whole picture data for pictures 
PC1, PC2, PC3, with high efficiency into digital 
data of much less quantity than the picture data com- 
pared with the case in transmitting the picture data, 
and send the digital data to a transmission path. 

Fig. 2 shows how a picture sequence is intra 
frame coded or inter frame coded and transformed 
into three types of frames (macrobtock) such as an in- 
tra frame predicted frame, forward inter frame predict- 
ed frame, and forward, backward, bidirectional inter 
frame predicted frame. In Fig. 2, 15 frame periods 
(frames FO to F14) are used as one unit for coding. 

For this example, the frame F2 is intra frame cod- 
ed and is called "an intra picture". The frames F5 t F8, 
F11 , and F14 are predicted only by the frames F2, F5, 
F8, and F11 located ahead of the above frames re- 
spectively, and are called "predicted pictures". 

The remaining frames FO, F1, F3, F4, F6, F7 t F9, 
F10, F12, and F13 are predicted by frames located 
ahead of the above frames, those located behind the 
above frames, or those located at the both sides of 
the above frames as well as inter frame coded, and 
are called "bidirectional pictures". 

Fig. 3 shows a video signal transmission system. 
The video signal transmission system 1 has a coding 
d vice 1A for transmitting data and a decoding device 
1 B for receiving coded data. 

After the coding device 1 A converts an input vid- 
eo signal VD via a preprocessing circuit 2 into a lumi- 
nance signal SY and color drff r nee signal SC, these 
are convert d into an 8-bit digital luminance signal DY 
and color difference signal DC by the analog to digital 



conversion circuits 3 and 4. 

Then, in the coding device 1A the digital lumi- 
nance signal DY and digital color difference signal DC 
are written into a luminance signal frame memory 5A 

5 and color difference signal frame memory 5B of a 
frame memory 5, respectively, and thereafter the pic- 
ture data is transformed from a frame format into a 
block format in a format conversion circuit 6. 

In the coding device 1A the picture data trans- 

10 formed into the block format is input to an encoder 7, 
and the picture data is high efficiently coded to gen- 
erate a bit stream which is transmitted to a decoding 
device 1B via a communication path and a recording 
media 8. 

15 The encoder 7 is composed of a hybrid coder 7A 

for discrete cosine transforming the intra frame or in- 
ter frame coded (forward, backward, or both way pre- 
dicted) picture data, and thereafter quantizing the pic- 
ture data, as well as a variable length coding appara- 

20 tus 7B for variable length coding and outputting the 
quantized data. 

The decoding device 1B decodes the bit stream 
taken from the recording media 8 by performing an in- 
nersive procedure of the coding device 1A. That is. 

25 the decoding device 1B inverse transforms the bit 
stream high efficiency coded by a decoder 9 so as to 
restore it, and transforms it from the block format into 
the frame format in a format transform circuit 10. 
Then, in the decoding device 1B the digital lumi- 

30 nance signal DY and digital color difference signal DC 
transformed into the frame format are written into a lu- 
minance signal frame memory 11 A and a color differ- 
ence signal frame memory 11B, respectively. 

Thereafter, in the decoding device 1 B, the above 

35 signals are converted into analog signal SY and SC 
via digital to analog conversion circuits 12 and 1 3, and 
the decoding device 1B inputs the signal to a post 
processing circuit 14 to obtain an output picture and 
output it as an output video signal VO. 

40 The video signal transmission system 1 is consti- 

tuted so as to transmit or receive data through a ser- 
ies of the processes. 

In this connection, the video signal transmission 
system 1 processes the data of a frame picture in the 

45 unit shown in Figs. 4 A to 4C. First, the data for a frame 
picture is divided into N slices as shown in Fig. 4A. 
Each slice includes M macroblocks as shown in Fig. 
4B and each macroblock includes luminance signal 
data Y1 to Y4 corresponding to 8 x 8 pixels and color 

50 difference data Cb and Cr corresponding to the total 
pixel data as shown in Fig. 4C. 

In this case, th pictur data stream in each slice 
is arranged so that picture data follows in macro- 
blocks, and also follows in microblocks in each mac- 

55 roblock in order of rast r scanning. 

The macroblock uses picture data (Y1 to Y4) for 
16x16 pixels following in the horizontal and vertical 
scanning directions for a luminance signal as one unit, 
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but uses one microbiock Cr or Cb assigned to the data 
for 1 6 x 16 pixels as an unit, because the data quantity 
is d creased and thereafter time base multiplexed for 
two color differenc signals corresponding to the pic- 
ture data (Y1 to Y4) for 16 x 16 pixels. 5 

For the coding device 1A, only one type of vari- 
able length coding (VLC) apparatus 7B of the encoder 
7 for high efficiency coding picture data is given as a 
conversion table (hereinafter referred to as "VLC ta- 
ble") used for variable length coding quantized data 10 
independent of the type of picture processed. For the 
encoder 7 based on MPEG1 (Moving Picture Experts 
Group 1), for example, the VLC table is constituted in . 
accordance with the quantized data generated 
through inter frame coding. 1 5 

However, the result of comparing the quantiza- 
tion data generated by inter frame coding with the 
quantized data generated by intra frame coding does 
not always have the same frequency distribution. 
That is, when considering the quantized data gener- 20 
ated by intra frame coding as a macroblock, it is very 
similar to the quantized data generated by inter frame 
coding in composition, but a considerably large differ- 
ence is found between them in the overall composi- 
tion (or inclination). Therefore, it is estimated that a 25 
high efficiency coding cannot be expected merely by 
directly applying the VLC table prepared correspond- 
ing to inter frame coding to the processing of the 
quantized data generated through intra frame coding. 

According to the invention there is provided a 30 
coding method for variable length coding quantized 
input data in predetermined blocks (e.g. in macro- 
blocks), in which input data is variable length coded 
by using any one of a plurality of given variable length 
coding tables 23C and 23D which is selected in accor- 35 
dance with the coding efficiency. 

Moreover, this invention uses a coding apparatus 
for variable length coding quantized input data in pre- 
determined blocks (e.g. in macroblocks), the coding 
apparatus comprising: a plurality of variable length ao 
coding tables 23C and 23D assigned with different 
sign lengths; a coded information generating means 
22 for designating the change of variable length cod- 
ing tables used for variable length coding input data 
in accordance with a coding efficiency: and a coding 45 
means 23A and 23B for variable length coding input 
data by using any one of the variable length coding 
tables 23C and 23D, which is selected in accordance 
with the coded information outputted from the coded 
information generating means 22. 50 

Furthermore, this invention uses a decoding 
m thod for variable length decoding coded data in- 
putted by a r cording m dium or transmission path in 
pred termin d blocks (e.g. in macroblocks), in which 
a variable I ngth coding table identical to the variable 55 
length coding tabl 23C or 23D used when cod d 
data is generated is selected among a plurality of giv- 
en variable length coding tables 32C and 32D and 



coded data is variable length decoded by using the 
variable length coding table 32C or 32D. 

Furthermore, this invention uses a decoding ap- 
paratus for variable length decoding coded data in- 
putted through a recording medium or transmission 
path in predetermined blocks (e.g. in macroblocks), 
th6 decoding apparatus comprising: a plurality of va- 
riable length coding tables 32C and 32D assigned to 
different sign lengths; a switching means 32B for se- 
lecting a variable length coding table identical to the 
variable length coding table 23C or 23D used when 
the coded data is generated is selected among a plur- 
ality of variable length coding tables 32C and 32D in 
accordance with coded information extracted from 
coded data; and decoding means 32A for variable 
length decoding coded data by using the variable 
length coding table 32C or 32D selected by the 
switching means 32B. 

It is possible to further improve the variable 
length coding efficiency compared with the case in 
specifying one variable length coding table by using 
the variable length coding table 23C or 23D selected 
among a plurality of given variable length coding ta- 
bles 23C and 23D based on a coding efficiency and 

variahlf* lonnth rnHinn Inm if rloH I trts-* n tKor«k„ ~ 

O-" —"-"-«• 'jj .i.^i-fc wmvu. wpvsi 1 iiiQiCbrjr yen- 

erating information content identical to the content 
when specifying one variable length coding table, it is 
possible to process data quantized in a smaller quan- 
tization size and further improve the quality of infor- 
mation to be transmitted as coded data. 

It is possible further to improve the variable 
length coding efficiency by variable length coding in- 
put data with a variable length coding table selected 
in accordance with the coding efficiency among a 
plurality of variable length coding tables prepared 
compared with the case for performing variable 
length coding with only one variable length coding ta- 
ble. 

Thereby, when generating the information con- 
tent equal to that generated by using one variabl 
length coding table, it is possible to select a smaller 
quantized size and further improve the quality of the 
information transmitted as coded data. 

Embodiments of the invention described below 
provide a coding method and system with a higher 
coding efficiency than conventional ones, and a de- 
coding method and system for decoding the thus- 
coded data. 

The invention will now be further described, by 
way of illustrative and non-Hmiting example, with ref- 
erence to the accompanying drawings, in which iik 
parts are designated by like refer nee numerals or 
charact rs throughout and in which: 

Figs. 1A and 1B are schematic diagrams for ex- 
plaining inter frame coding; 
Fig. 2 is a sch matic diagram showing a picture 
structure in a picture sequ nee; 
Fig. 3 is a block diagram showing a video signal 
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transmission system; 

Figs. 4A to 4C are schematic diagrams showing 

a hierarchical structure of picture data; 

Fig. 5 is a block diagram showing a coding system 

using the variable length coding according to an 

embodiment of this invention; 

Fig. 6 is a flow chart for explaining a variable 

length coding procedure; 

Fig. 7 is a block diagram showing a hybrid coder; 
Fig. 8 is a frequency distribution diagram showing 
the generation frequency of quantized data ob- 
tained through intra frame coding; 
Fig. 9 is a frequency distribution diagram showing 
the generation frequency of quantized data ob- 
tained by inter frame coding; 
Figs. 1 0A and 10D are schematic diagrams show- 
ing a DCT coefficient scanning procedure; 
Figs. 11 A to 11C are schematic diagrams for ex- 
plaining quantized data having two-dimensional 
data of runs and levels; 

Fig. 12 is a chart showing a VLC table for inter 
frame coding; 

Fig. 13 is a chart showing assignment of bits in 
accordance with a VLC table for inter frame cod- 
ing; 

Fig. 14 is a chart showing a VLC table for intra 
frame coding; 

Fig. 15 is a chart showing assignment of bits in 
accordance with a VLC table for intra frame cod- 
ing; 

Fig. 16 is a block diagram showing a decoding 
system using variable length decoding according 
to an embodiment of this invention; 
Fig. 17 is a block diagram showing a hybrid de- 
coder; 

Fig. 18 is a block diagram showing a coding sys- 
tem using variable length coding according to an 
embodiment of this invention; 
Fig. 19 is a block diagram showing a table change 
signal generator, 

Fig. 20 is a flow chart for explaining a variable 
length coding procedure; and 
Fig. 21 is a block diagram showing a decoding ap- 
paratus using variable length coding according to 
an embodiment of this invention. 
Preferred embodiments of this invention will be 
described with reference to the accompanying draw- 
ings: 

(1) First embodiment 

(1-1) Constitution of encoder 

In Fig. 5, 20 r presents an encoder constituting a 
motion picture coding apparatus as a whol , which is 
characterized by using two types of variable length 
coding tables corresponding to the coding syst m to 
improve a coding efficiency. A frame m mory 21 



fetches input picture signals SO in order and supplies 
picture data S1 to be processed at present in macro- 
blocks to a hybrid coder 22. 

The hybrid coder 22 movement compensation 

5 predicted codes the picture data S1 and thereafter 
hybrid codes the picture data S1 by discrete cosine 
transform (DCT). 

The hybrid coder 22 supplies picture information 
to a variable length coding (VLC) section 23 as a 

10 quantized signal S2 and control information to it as a 
picture coding control signal S3 among processed re- 
sults obtained by hybrid coding. 

The VLC section 23 inputs the quantized signal 
S2 to a VLC circuit 23A and the picture coding control 

15 signal S3 to a VLC circuit 23A and a table changer 
23B. In this connection, the quantized signal S2 has 
a DCT coefficient which is a movement compensation 
predicted error signal in a macroblock layer and run 
length information and the picture coding control sig- 

20 nal S3 has control information including movement 
vector, macroblock type (presence or absence of 
movement compensation mode and DCT coefficient), 
and DCT mode. 

The VLC circuit 23A variable length codes the 

25 DCT coefficient inputted as the quantized signal S2 
based on two conversion tables (that is, the VLC ta- 
ble 23C for intra frame coding and the VLC table 23D 
for inter frame coding). 

The movement compensation mode signal input- 

30 ted as the picture coding control signal S3 is used to 
exchange the two conversion tables in the VLC circuit 
23A. Exchanging the two conversion tables is descri- 
bed below by referring to Fig. 6. 

First, the VLC circuit 23A starts with a processing 

35 routine RTO. When the VLC circuit 23A inputs the 
DCT coefficient and run length information from the 
hybrid coder 22 in the step SP1 , it judges if the picture 
data to be processed is an intra frame coding mode 
or not in the next step SP2. 

40 If so, the VLC circuit 23 A proceeds to the step 

SP3 and references the VLC table 23C for intraframe 
coding to obtain corresponding variable length coded 
data. If not, the VLC circuit 23B proceeds to the step 
SP4 and references the VLC table 23D for inter frame 

45 coding to obtain corresponding variable length cod d 
data. 

Then, the VLC circuit 23A transfers the variable 
length coded data obtained by reference to each table 
to a buffer memory 24 for transmission (step SP5) 
so and then terminates all processing (step SP6). 

Thereby, the VLC circuit 23 variable length codes 
quantized data together with a quantizing step size 
and macroblock (movement compensation mode) 
and supplies it to the transmission buffer memory 24 
55 as transmission data. 

The buffer memory 24 stores the data thus vari- 
abl I ngth coded and thereafter outputs the data as 
a bit stream at a certain transmission rate. 
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In this case, the buffer memory 24 returns quan- 
tization control signals in macrobtocks to a quantiza- 
tion circuit based on the data quantity remaining in 
the memory so as to control the quantizing step size. 
The buffer memory 24 thereby adjusts the data quan- 5 
tity generated as a bit stream to keep the data in the 
memory at a proper quantity (a data quantity causing 
no overflow or underflow). 

That is, when the quantity of remaining data in- 
creases up to the allowable upper limit, the buffer 10 
memory 24 decreases the quantity of quantized data 
by increasing the quantizing step size of the quanti- 
zation circuit with the quantization control signal. 
However, when the quantity of remaining data de- 
creases down to the allowable lower limit, the buffer 15 
memory 24 increases the quantity of quantized data 
by decreasing the quantizing step size of the quanti- 
zation circuit with the quantization control signal. 

(1-2) Constitution of hybrid coder 20 

Signal processing of the hybrid coder 22 is de- 
scribed below by referring to Fig. 7. 

The hybrid coder 22 supplies a block format pic- 
ture to a motion vector detection circuit 22A through 25 
the frame memory 21 to detect a motion vector. 

The motion vector detection circuit 22A, as al- 
ready described in Fig. 2, generates an interpolation 
picture by using a non interpolation frame (that is, intra 
frame predicted frame) as a predicted picture and us- 30 
ing the detected motion vector. 

In this case, the motion vector detection circuit 
22A processes the picture data for each frame as an 
intra picture, predicted picture, or bidirectional picture 
based on previously set predetermined sequences. 35 

The frame memory 21 stores the picture data for 
each frame in a front original picture memory section 
21 A, reference original picture memory section 21 B, 
or rear original picture memory section 21 C in accor- 
dance to these predetermined sequences. 40 

The motion vector detection circuit 22A reads a 
reference picture DATA and predicted picture DATA 
(that is, front original picture and rear original picture) 
from the original picture memory sections 21A to 21C 
to detect the motion vector for each block. At this time, 45 
the motion vector detection circuit 22A uses the mini- 
mum value among the absolute value sums of inter 
frame differences. 

The motion vector detection circuit 22A transmits 
the absolute value sums of inter frame differences ob- 50 
tained in blocks to an intra frame/forward/bidirectional 
predicted judgment circuit 22B. 

The intra frame/forward/bid ir ctional predicted 
judgm nt circuit 22B d termines th frame type of a 
reference block along with the absolute value sums 55 
and exchanges th processes of an operation section 
22C in macroblocks in along with the d termined 
frame type. 



In the case of an intra frame coded frame, the op- 
eration section 22C outputs a picture inputted from 
the motion vector detection circuit 22A through a 
switching circuit 22C3. 

For a forward pr dieted coded frame or bidirec- 
tional predicted coded frame, a subtracter 22C1 or 
22C2 generates intra frame coded data from a pre- 
dicted picture and outputs the differential data 
through the switching circuit 22C3. 

After a discrete cosine transform (DCT) circuit 
22D inputs intra frame coded or interframe coded pic- 
ture data from a computing unit 22C, it discrete cosine 
transforms the input picture data or differential data 
in blocks by using two dimensional correlation of vid- 
eo signals. 

A quantizer 22E quantizes the DCT coefficient 
obtained by transforming a macroblock and a quan- 
tizing step size determined for each slice and outputs 
the quantized data obtained through the quantization 
at an output terminal to a variable length coding sec- 
tion 23. 

In this connection, the quantizing step size of the 
quantizer 22E is determined to a value so as not to ex- 
ceed the capacity of a transmission by returning the 
remaining capacity of the transmission buffer. The 
quantizing step size is also outputted to the variable 
length coding circuit 23. 

The hybrid coder 22 inputs the quantized data 
and quantizing step size to be outputted to the vari- 
able length coding section 23 to an inverse quantizer 
22F to start local decoding. 

The inverse quantizer 22E decodes the data to 
be transformed in the quantizer 22E by inversely 
quantizing the quantized data sent from the quantiz r 
22E into a representative value and transforming it 
into inversely quantized data and supplies the in- 
versely quantized data to an inverse discrete cosine 
transform (I DCT) circuit 22G. 

The IDCT circuit 22G transforms the inversely 
quantized data decoded by the inverse quantization 
circuit 22F into decoded picture data by a transform 
processing inverse to that of the DCT circuit 22D and 
inputs it to a computing unit 22H. 

Data same as the predicted picture data supplied 
to the computing unit 22C is supplied to the comput- 
ing unit 22H so as to add the predicted picture data 
outputted by a motion compensation circuit 22J to the 
differential data outputted by the IDCT circuit 22G. 

Thereby, picture data for the original (restored) 
predicted picture and picture data for the original (re- 
stored) intra picture are obtained and stored in a rear 
original picture memory section 2211 and a front orig- 
inal picture memory section 22I2 of a frame memory 
22I r spectiv ly. 

After th picture data is stored in the rear original 
picture memory section 2211 and front original picture 
memory section 22 12 respectiv ly, the motion vector 
detection circuit 22A starts the next picture process- 
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ing. 

The motion compensation circuit 22J generates 
predicted picture data by using the picture data stored 
in the original picture memory sections 2211 and 22I2 
in accordance with the fram type determined by the 
intra frame/forward/bidirectional predicted judgment 
circuit 22B. 

That is, for the backward predicted frame, the 
motion compensation circuit 22J sends a read ad- 
dress deviated by a value corresponding to a motion 
vector from a position corresponding to the position 
of a block to be inputted to the operating unit 22C to 
the rear original picture memory section 2211 and 
generates predicted picture data. 

For the bidirectional predicted frame, however, 
the motion compensation circuit 22J sends a read ad- 
dress deviated by a value corresponding to a motion 
vector from a position corresponding to the position 
of a block to be inputted to the operating unit 22C to 
the rear original picture memory sections 2211 and 
22I2 and generates predicted picture data. 

(1-3) Variable length coding table 

The following is the description of two types of 
coding tables prepared to efficiently code the quan- 
tized data S2 to be inputted from the hybrid coder 22 
to the VLC apparatus 23. 

The VLC apparatus 23 earns the coding efficien- 
cy in varying the sign length of quantized data coded 
in accordance with a combination of 0-run length (run 
length) with a quantized data level depending on a 
generation frequency. For this example, the VLC ta- 
ble 23C for intra frame coding and VLC table 23D for 
inter frame coding are prepared. 

This is because intra frame coded quantized data 
and inter frame coded quantized data are different 
from each other in the frequency distributions of a run 
length and level. 

That is, the intra frame coded quantized data is 
concentrated on a low frequency component because 
intra frame coding discrete cosine transforms a pat- 
tern itself. Therefore, a run length tends to compara- 
tively decreases and a level tends to comparatively 
rises as shown in Fig. 8. 

For the inter frame coded quantized data is scat- 
tered up to a high frequency component because a 
difference with the pattern of other frame is discrete 
cosine transformed as shown in Fig. 9. Therefore, a 
run length tends to comparatively increases and a 
level tends to comparatively lower. 

The VLC table 23C for intra frame coding and 
VLC table 23D for inter frame coding are the coding 
tables assign d with signs so that the sign length of 
quantized data can be efficiently decreased by re- 
flecting the above composition. 

The quantized data S2 to be variable length cod- 
ed here is obtained by quantizing a DCT coefficient 



transformed by the scan procedure shown in Figs. 
1 0A to 1 0D no matter when the data is coded by any 
coding method. 

In general, the quantized data S2 is formed in ac- 

5 cordance with a combination of 0-run length with a 
quantized data level (that is, run and level) as shown 
in Figs. 11 A to 11 C. However, when the entire quan- 
tized data remaining without being scanned is 0, it is 
notified to the quantized data S2 by sending end of 

10 block (EOB) code that there is no level other than 0 
up to finally coded quantized data. 

Figs. 12 and 13 show the VLC table 23D for inter 
frame coding prepared for the quantized data S2 ex- 
pressed by the format and Figs. 14 and 15 show the 

15 VLC table 23C for intra frame coding. In Figs. 12 and 
14, the central column shows run length values and 
the right column shows level values. And, codes as- 
signed to combinations of run lengths with levels are 
shown in the left column. 

20 From Fig. 12, it is found that in case oMhe VLC 

table 23D for inter frame coding a sign with a small bit 
length is assigned to quantized data with a relatively 
high generation frequency and a large run length. 
This is also found from Fig. 13 showing the relation- 

25 ship between actually assigned sign length and com- 
bination of a run length with a level. 

In other words, this is also found from the fact 
that a sign with the length of 9 bits is assigned to the 
quantized data with the run length of "0" and the level 

30 of "5" but only a sign with the length of 6 bits is as- 
signed to the quantized data with the run length of "4" 
and the level of "1". 

In this connection, the bit length shown in Fig. 13 
is a value obtained by adding one bit to the code 

35 length of the VLC table 23D for inter frame coding 
shown in Fig. 12. 

The VLC table 23D for inter frame coding assigns 
a code with the sign length of 20 bits to quantized 
data of a combination with a relatively low generation 

40 frequency instead of assigning a sign so as to fixed 
length code the quantized data. In this case, six bits 
of the above 20 bits are assigned to "ESCAPE code", 
six bits of them are assigned to "run length", and eight 
bits of them are assigned to level". 

45 From Fig. 14, it is found that in case of the VLC 

table 23C for intra frame coding a sign with a small bit 
length is assigned to quantized data with a relatively 
high generation frequency and a low level. This is also 
found from Fig. 15 showing the relationship between 

50 each combination and a sign lengt h actually assigned 
to each combination. 

In other words, this is also found from the fact 
that a sign with the length of 6 bits is assigned to 
quantized data with the run length of "0" and the level 

55 of "5" but only a sign with the I ngth of 7 bits is as- 
signed to quantized data with the run length of "4" 
and the level of "1". 

Also in the case of Fig. 1 5, the bit lengt h assigned 
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for a combination of a run length with a level is a value 
obtained by adding one bit to the code length of the 
VLC table 23C for intra frame coding shown in Fig. 14. 

The VLC table 23C for intra frame coding assigns 
a code with the sign length of 22 bits to quantized 5 
data of a combination with a relatively low generation 
frequency instead of assigning a sign so as to fixed 
length code the quantized data. In this case, eight bits 
of the above 22 bits are assigned to "ESCAPE code", 
six bits of them are assigned to "run length", and eight 10 
bits of them are assigned to "level". 

(1-4) Constitution of decoder 

Thefollowing is the description of the constitution 15 
of a decoder for restoring a motion picture from a bit 
stream coded by the encoder 20 and transmitted 
through a transmission path or media. 

In Fig. 16, 30 represents a decoder constituting a 
motion picture decoding device as a whole. The de- 20 
coder 30 temporarily stores a bit stream S10 inputted 
from media or the like in a receiving buffer memory 
31 and successively inputs it to an inverse variable 
length coding (inverse VLC) section 32 at a predeter- 
mined timing. 95 

The inverse VLC section 32 inverse VLC codes 
the information related to movement compensation of 
header information included in a bit stream S11 with 
an inverse VLC circuit 32A and thereafter supplies re- 
stored results to a table changer 32B as a motion com- 30 
pensation mode signal S13. 

In this case, the table changer 32B selects the 
VLC table 32C for intra frame coding as an inverse 
transform table when the motion compensation mode 
of a macroblock to be processed is a intra frame cod- 35 
ing mode but selects the VLC table 32D for inter frame 
coding as an inverse transform table when the motion 
compensation mode is an inter frame coding mode. 

The inverse VLC circuit 32A refers to the inverse 
coding table thus selected by the table changer 32B 40 
to inverse VLC code a DCT coefficient and run length 
inputted as the bit stream S11. 

The hybrid decoder 33 inputs from inverse VLC 
circuit 32A the DCT coefficient and run length infor- 
mation of the motion compensation predict error sig- 45 
nal obtained for the macroblock layer and decoded 
data S12 of the motion vector and motion compensa- 
tion mode, etc. 

A hybrid decoder 33 restores picture data by de- 
coding high efficiently coded data according to a pro- 50 
cedure reverse to that of the hybrid coder 22. 

(1-5) Constitution of hybrid decoder 

The internal structure of the hybrid d coder 33 is 55 
d scribed below by referring to Fig. 17. 

The hybrid decoder 33 sends a quantized step 
and picture data of the variable length coded data in- 



putted from the inverse VLC section 32 to an inverse 
quantizer 33A and also sends a motion vector, pre- 
dicted mode, predicted flag, and DCT flag to a motion 
compensation circuit 33D. 

The inverse quantizer 33A inversely quantizes 
the picture data supplied from the inverse VLC appa- 
ratus 32 in accordance with the quantized step simi- 
larly supplied from the inverse VLC section 32 and 
supplies the data to an invert discrete cosine trans- 
form (IDCT) circuit 33B. 

The IDCT circuit 33B inverse DCT transforms a 
DCT coefficient inputted from the inverse quantizer 
33A to supply it to an operating unit 33C. 

The operating unit 33C restores an original pic- 
ture by adding the output of the IDCT circuit 33B to 
the data motion compensated by the motion compen- 
sation circuit 33D. 

The picture data thus outputted from the comput- 
ing unit 33C is outputted to an output terminal and 
also it is stored in a frame memory 33E. 

The frame memory 33E is provided with a front 
predicted picture memory section 33E1 and a rear 
predicted picture memory section 33E2. The front 
predicted picture memory section 33E1 stores a pre- 

HintoH r»ir»fi ira onrl <a« infr-o niAhim *«■»..■ _ i_ : 
— f . w-w. w m» 1 iiiuh piv/iuic iui yci 101 dill iy ci Ul- 

directional picture and the rear predicted picture 
memory section 33E2 stores a predicted picture for 
generating a predict picture or bidirectional picture. 

The motion compensation circuit 33D compen- 
sates the motion of picture data stored in the front pr - 
dieted picture section 33E1 and rear predicted picture 
section 33E2 and sends the data to the computing 
unit 33C in accordance with a mode of the picture 
data to be decoded. 

(1-6) Coding and decoding 

The following is the description of coding by a mo- 
tion picture coding section for successively coding 
motion pictures in accordance with a predetermined 
procedure with the above constitution. 

The motion picture coding section stores a motion 
picture converted into digital data by a pre-processing 
circuit 2 or the like in the encoder 20 through a frame 
memory 21 to high efficiently code the data with the 
encoder 20. 

The encoder 20 hybrid codes the picture data 
read from the frame memory 21 with the hybrid coder 
22 and sends the quantized data which is motion 
compensated and DCT transformed and thereafter 
quantized to the VLC section 23. 

The VLC section 23 variable length codes a cod- 
ing table used for variable length coding by changing 
the VLC table 23C for intra frame coding and the VLC 
table 23D for inter frame coding correspondingly to 
two types of coding modes. 

Thus, the VLC section 23 variable length codes 
a transmission picture by using the VLC table 23C for 
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intra frame coding for reflecting the genernerneration 
frequency of intra frame coded quantized data when 
the picture comprises intra frame coded picture data 
but variable length codes the picture by using the 
VLC table 23D for inter frame coding for reflecting the 
generation frequency of inter frame coded quantized 
data when the picture is composed of inter frame cod- 
ed picture data. 

The above configuration makes it possible to fur- 
ther decrease the bit length of transformed and 
thereafter outputted picture data even though it is the 
quantized data coded by any coding system, com- 
pared with the existing case in which variable length 
coding is performed by using a VLC table for inter 
frame coding. 

Thereby, it is possible to decrease the transmis- 
sion rate for transmitting the same motion picture 
compared with the existing one and decrease the 
quantized size by a value equivalent to the decrease 
of the transmission rate. Thus, for the coding system 
of this embodiment, it is possible to increase informa- 
tion content to be assigned to each pixel by a value 
equivalent to the decrease of the quantized size and 
further improve the picture quality compared with the 
existing one. 

Moreover, because it is possible to use an exist- 
ing coding table for the VLC table 23D for inter frame 
coding of this embodiment, the compatibility with ex- 
isting models is also compensated. 

In this connection, because the information 
about whether to perform intra frame or inter frame 
coding every macroblock is sent together with picture 
data even for an existing coding system, it is not nec- 
essary to newly add the information about which VLC 
table should be used to header information. 

(2) Second Embodiment 

(2-1) Constitution of Encoder 

In Fig. 18 in which a portion corresponding to that 
in Fig. 5 is provided with the same symbol, symbol 40 
represents an encoder constituting a motion picture 
coding system as a whole which has the same con- 
stitution as the encoder in Fig. 5 except a table 
change signal generator 41 for selecting a coding ta- 
ble requiring a small bit length regardless of a predict- 
ed coding system (frame type) by considering the in- 
formation content generated by using each coding ta- 
ble in addition to the fact that two types of transform 
tables corresponding to a coding system are pre- 
pared as variable length coding tables. 

The table change signal generator 41 has the 
constitution shown in Fig. 1 9. 

Th table change signal generator 41 has two 
g nerated informati n counting sections t calculate 
the information content generated by using each 
transform tabl before starting variable length coding 



with the VLC apparatus 23. 

The table change signal generator 41 inputs a 
DCT coefficient and run length information to two 
VLC circuits 41Aand 418 from the hybrid coder 22 to 
5 variable length code quantized data with the VLC cir- 
cuits 41 A and 41 B. 

In this case, the VLC circuit 41 Auses the VLC ta- 
ble 23C for intra frame coding as a coding table and 
the VLC circuit 41 B uses the VLC table 23D for inter 

10 frame coding as a transform table. 

The data variable length coded by the VLC circuit 
41A and the data variable length coded by the VLC 
circuit 41 B are sent to the number of bits counters 
41C and 41D respectively where the number of bits 

15 representing the generated information content is 
counted. In this case, the number of bits counters 41 C 
and 41 D update the generated information content 
stored in memories 41 C2 and 41 D2 by sequentially 
adding the number of bits of newly coded variable 

20 length data to the number of bits stored in the mem- 
ories 41C2 and 41D2 with adders 41C1 and 41D1. 

A comparator 41 E sends a data request signal SR 
to the memories 41 C2 and 41 D2 in order to compare 
the information content generated in macroblocks 

25 and compares the number of bits stored in the mem- 
ory 41 C2 with that stored in the memory 41 D2 at the 
timing when a response signal SA representing that 
the number of bits of all macroblocks is requested 
from the memories 41C2 and 41D2 is inputted. 

30 The comparator 41 E outputs the comparison re- 

sult to the VLC circuit 23A and table changer 23B of 
the VLC apparatus 23 as a table change signal. 
Thereby, the comparator 41 E shows the VLC circuit 
23A a coding table with which variable length coding 

35 should be performed and changes the connection of 
the table changer 23B. 

A series of these processing by an encoder 40 is 
described by referring to Fig. 20. 

The encoder 40 starts the processing of a proc- 

40 essing routine RT1 when the processing by the hybrid 
coder 22 terminates and quantized data is obtained 
and then starts the processing in the steps SP10 and 
SP11. In the steps SP10 and SP11, the table change 
signal generator 41 counts the generated number of 

45 bits N1 when variable length coding quantized data 
by using the VLC table 23C for intra frame coding and 
the generated number of bits N2 when variable length 
coding quantized data by using the VLC table 23Dfor 
inter frame coding in order. 

50 When counting of the generated number of bits 

N1 and the generated number of bits N2 for each 
macroblock terminat s, the table change signal gen- 
erator 41 starts the n xt step SP12 to store the num- 
ber of bits in a memory and det r mine a counted val- 

55 ue. 

The table change signal generator 41 decides a 
coding table for coding with which the generated 
number of bits is minimized in the next step SP13 (ac- 
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tually, comparison is performed by the comparator 
41 E). 

When the table change signal generator 41 de- 
cides that the generated number of bits is minimized 
by the VLC table 23C for intra frame coding, it starts 
the step SP14 to control the table changer 23B so as 
to refer to the VLC table 23C for intra frame coding ac- 
cording to a change signal. 

However, when the table change signal genera- 
tor 41 decides that the generated number of bits is 
minimized by the VLC table 23D for inter frame cod- 
ing, it starts the step SP15 to control the table 
changer 23B so as to refer to the VLC table 23D for 
inter frame coding according to a change signal. 

When these changes terminate, the step SP16 is 
started and the^LC circuit 23A starts variable length 
coding the quantized data having a DCT coefficient 
and a run length fetched from the hybrid coder 22. 

Moreover, the VLC circuit 23A adds a flag for des- 
ignating the coding table used for variable length cod- 
ing in the step SP17 to header information as control 
data. 

In the step SP18, the VLC circuit 23A transfers 
variable length coded picture data to a buffer memory 
24 for transmission together with the header informa- 
tion to temporarily store them and then writes the 
stored data in a transmission path or media from the 
buffer memory 24 to terminate the above coding. 

(2-2) Constitution of Decoder 

The constitution of a decoder for decoding picture 
data high efficiently coded by the encoder 40 is de- 
scribed below by referring to Fig. 21. In Fig. 21 in 
which a portion corresponding to that in Fig. 16 is pro- 
vided with the same symbol, symbol 50 represents a 
decoder which has the same constitution as that in 
Fig. 16 except that an inverse VLC circuit 51 is used 
instead of the inverse VLC circuit 32A of the inverse 
VLC section 32. 

The inverse VLC circuit 51 inversely VLC codes 
header information to extract a flag showing the cod- 
ing table selected in variable length coding among the 
transform results and use it to change the table 
changer 32B. 

As the result of the above change, the inverse 
VLC circuit 51 refers to the transform table selected 
by the table changer 32B to inversely VLC transform 
a DCT coefficient and run length information inputted 
as bit streams. 

(2-3) Coding and Decoding 

The following is th d scription of coding p r- 
form d by a motion picture coding system for succes- 
sively coding motion pictures according to a pr deter- 
mined procedure with the above constitution. 

The motion picture coding system stores a motion 



picture converted into digital data by the pre-process- 
ing circuit 2 or the like in the encoder 40 through the 
frame memory 21 to high efficiently code the data 
with the encoder 40. 
5 The encoder 40 hybrid codes the picture data 

read from the frame memory 21 with the hybrid coder 
22 and sends the quantized data which is motion 
compensated and DCT transformed and thereafter 
quantized to the VLC apparatus 23. 
10 In this case, the quantized data same as that sent 

to the VLC apparatus 23 is also inputted to the table 
change signal generator 41 to decide a coding tabl 
form minimizing the generated information content 
for variable length coding. 

15 This is because intra frame coded quantized data 

of a macroblock includes data with a composition 
very close to inter frame coded quantized data of the 
macroblock and data with a composition not very 
close to it and therefore intra frame coded quantized 

20 data cannot be always transformed into smaller num- 
ber of bits by using a VLC table for intra frame coding. 

Therefore, intra frame coded quantized data of a 
macroblock with a composition very close to inter 
frame coded quantized data of the macroblock is va- 

25 riable length coded b" 

ijcjnn the VLC table 23D for in- 
ter frame coding and macroblocks other than the 
above macroblock are variable length coded by using 
the VLC table 23C for intra frame coding. Thereby, it 
is possible to further improve the coding efficiencyfor 

30 variable length coding. 

In this case, because a coding table used for va- 
riable length coding does not depend on an intra 
frame coding system or inter frame coding system in 
general, it is necessary to code and transmit the in- 

35 formation showing which VLC table is used for vari- 
able length coding together with variable length cod- 
ed data. 

Therefore, a VLC table formed by adding one bit 
to each macroblock is used. 

40 The above constitution makes it possible to real- 

ize a variable length coding system with higher vari- 
able length coding efficiency than the existing one by 
variable length coding data with a composition close 
to quantized data generated through inter frame cod- 

45 ing by using a VLC table for inter frame coding even 
when variable length coding quantized data generat- 
ed through intra frame coding and selecting either of 
two coding tables for further decreasing generated in- 
formation content for each block. 

so Therefore, it is possible to decrease the transmis- 

sion rate for transmitting the same motion picture 
compared with the existing one and decrease the 
quantizing step size by a value equivalent to the de- 
crease of the coding rate. Thus, the coding system of 

55 this embodiment makes it possible to increase the in- 
formation content to be assigned to each pixel com- 
pared with an existing coding system which has only 
one VLC table and further improve the picture quality. 
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Moreover, though this variable length coding sys- 
tem requires additional information of one bit for each 
macroblock, generated information content can be 
decreased because it is possible to make the de- 
crease rate of the number of bits larger than the in- 
crease rate of the number of bits for additional infor- 
mation by properly changing VLC tables. 

(3) Other Embodiments 

In the above embodiment, a video signal trans- 
mission system used for a video conference system 
and visual telephone system is described. However, 
this invention is not restricted to the system but it can 
be widely applied to a system using various types of 
recording and reproducing systems and a system us- 
ing transmitting and receiving systems. 

Moreover, in the above embodiment, a case is 
described in which the quantized data obtained from 
picture data is variable length coded. However, this 
invention is not restricted to the case but it can be 
widely applied to cases for variable length coding the 
quantized data obtained from other data such as 
voice data. 

Furthermore, in the above embodiment a case 
is described in which two variable length coding ta- 
bles are prepared correspondingly to data which is in- 
tra framed coded and thereafter quantized and data 
which is inter frame coded and thereafter quantized. 
However, this invention is not restricted to the case 
but it can be applied to a case in which three or more 
variable length coding tables assigned with different 
sign lengths are prepared. 

Furthermore, in the above embodiment, a case 
is described in which intra frame coded or inter frame 
coded picture data is discrete cosine transformed. 
However, this invention is not restricted to the case 
but it can be widely applied to cases for transforming 
the data with any other orthogonal coding system. 

Furthermore, in the above embodiment, a case 
is described in which quantized data is variable length 
coded in macroblocks. 

However, this invention is not restricted to the 
case but it can be widely applied to cases for using a 
block with any size as the processing unit 



Claims 

1. A coding method for variable length coding quan- 
tized input data in predetermined blocks, com- 
prising the step of: 

variable length coding the input data by 
using a variable I ngth coding tab! selected 
among a plurality of variable length coding tables 
(23C, 23D) prepared based on coding eff ici ncy. 

2. A coding system for variable length coding quan- 



tized input data in predetermined blocks, com- 
prising: 

a plurality of variable length coding tables 
(23C, 23D) assigned with different sign lengths; 
5 a coded information generating means 

(22) for designating change in variable length 
coding tables used for variable length coding the 
input data based on with coding efficiency; and 

coding means (23A, 23B) for variable 
w length coding the input data by using a variable 

length coding table (23C, 23D) selected by coded 
information outputted from the coded information 
generating means. 

15 3. A decoding method for decoding coded data in- 
putted through a recording medium or transmis- 
sion path in predetermined blocks, comprising 
the step of: 

selecting a variable length coding table 

20 identical to the variable length coding (23C, 23D) 

used when the coded data is generated among a 
plurality of variable length coding tables (32C, 
32D) prepared and variable length decoding the 
encoded data by using the variable length coding 

25 table (32C, 32D). 

4. A decoding system for variable length decoding 
coded data inputted through a recording medium 
or transmission path in predetermined blocks, 

30 comprising: 

a switching means (32B) for selecting a va- 
riable length coding table (32C, 32D) identical to 
variable length coding (23C, 23D) used when th 
coded data is generated in accordance with cod- 

35 ed information extracted from the coded data 

among the variable length coding tables (32C, 
32D); and 

a decoding means (32A) for variable 
length decoding the coded data by using a vari- 
40 able length coding table (32C, 32D) selected by 

the switching means (32B). 

5. A coding system according to claim 2, wherein in- 
put data is data motion compensated and there- 

45 after quantized, and the coded information is in- 

formation generated based on a predictive cod- 
ing system. 

6. A decoding system according to claim 4, wherein; 
50 input data is data movement compensated 

and thereafter quantized, and the coded informa- 
tion is information gen rated bas dona predic- 
tiv coding syst m. 

55 7. A coding method according to claim 1, wherein: 
a variable length coding table is selected based 
on information content gen rated when variable 
length coding the input data by using each of the 
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prepared variable length coding tables without 
depending on a variable length coding table cor- 
responding to a predictive coding system. 

8. A coding system according to claim 2, wherein: 

the coded information generating means 
designates a variable length coding table for va- 
riable length coding the input data based on the 
information content generated when variable 
length coding the input data by using each of the 
prepared variable length coding tables without 
depending on a variable length coding system 
corresponding to a predictive coding system. 

9. A coding method according to claim 1 , wherein: 
each piece of information content generated 
when variable length coding the input data by us- 
ing each of the variable length coding tables be- 
fore variable length coding the input data is ob- 
tained to select a variable length coding table for 
variable length coding the input data based on 
the result of comparing the generated pieces of 
information content 

10. A coding system according to claim 2 ? wherein: 

the coded information generating means 
obtains each piece of information content gener- 
ated when performing variable length coding by 
using each of the variable length coding tables 
before the coding means variable length codes 
the input data and designates a variable length 
coding table for variable length coding the input 
. data based on the result of comparing the gener- 
ated pieces of information content. 

11. A decoding method according to claim 3, where- 
in: 

a variable length coding table selected 
based on the generated information content is 
selected among the prepared variable length 
coding tables without depending on a variable 
length coding table corresponding to a predictive 
coding system to variable length decode the cod- 
ed data by using the variable length coding table. 

12. A decoding system according to claim 4, wherein: 

the switching means selects a variable 
length coding table for variable length coding the 
input data based on information content generat- 
ed when variable length coding the input data by 
using each of the variable length coding prepared 
tables without depending on a variable length 
coding system corresponding to a predictive cod- 
ing system. 

13. A coding method according to claim 3, wherein: 

a variable length coding tabl id ntical to 
the variable length coding table selected for cod- 



ing is selected among the prepared variable 
length coding tables before variable length cod- 
ing the input data to variable length decode the 
coded data by using the variable length coding 
5 table. 

14. Adecoding system according to claim 4, wherein; 

the switching means selects a variable 
length coding table identical to the variable 
10 length coding table selected for coding among 

the prepared variable length coding tables before 
the decoding means variable length decodes the 
input data and variable length decodes the coded 
- data by using the variable length coding table. 

15 

15. A coding system for variable length coding quan- 
tized data obtainable by the orthogonal transfor- 
mation of picture data predictively coded for each 
macroblock, and thereafter quantizing the pic- 

20 ture data, comprising: 

a first variable length coding table suitable 
for the generation frequency of data quantized af- 
ter intra frame coding; 

a second variable length coding table suit- 
es able for the generation frequency of data quan- 
tized after inter frame coding; 

a coded information generating means for 
designating change in the first and second vari- 
able length coding tables based on the fact that 
30 the quantized data is data intra frame coded or in- 

ter frame coded as predictive coding; and 

a means for variable length coding the 
quantized data by using a first or second variable 
length coding table selected by coded informa- 
35 tion outputted from the coded information gener- 

ating means. 

16. A decoding system for variable length decoding 
coded data inputted through a recording medium 

40 or transmission path in macroblocks, comprising: 

a first variable length coding table suitable 
for the generation frequency of data quantized af- 
ter intra frame coding; 

a second variable length coding table suit- 
es able for the generation frequency of data quan- 
tized after inter frame coding; 

a changing means for extracting a coding 
method selected for predictive coding from 
among coded data to select the first variable 
50 length coding table when the coded data is data 

intra frame coded and thereafter variable length 
coded, and the second variable length coding ta- 
ble wh n the cod d data is data inter frame coded 
and th reafter variable length coded; and 
55 a decoding means for variable I ngth de- 

coding the coded data by using the first or sec- 
ond variabl length coding table selected by the 
switching means. 
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1 

MOTION COMPENSATED VIDEO SIGNAL PROCESSING 

This invention relates to motion compensated video signal 
processing. 

Motion compensated video signal processing is used in 
applications such as television standards conversion, film standards 
conversion and conversion between video and film standards. 

In a motion compensated television standards converter, such as 
the converter described in UK patent application number GB-A-2 231 7^9. 
pairs of successive input images are processed to generate sets of 
motion vectors representing image motion between the pair of input 
images. The processing is carried out on discrete blocks of the 
images, so that each motion vector represents the inter-image motion of 
the content of a respective block. 

In the motion vector estimation process a correlation surface 
representing spatial correlation between blocks of the two input images 
is examined to detect points of maximum correlation. (The correlation 
surface actually represents the difference between the two input images 
so that the points of maximum correlation are in fact minima on the 
correlation surface, and will be referred to as such herein.) When a 
minimum has been detected, a motion vector is generated from the 
spatial position of the minimum in the correlation surface. Tests are 
performed to determine whether that minimum represents a significant 
peak in correlation with respect to the remainder of the correlation 
surface. If the minimum passes this test it is treated as being 
"valid", and a confidence flag associated with the motion vector is 
set. 

After the individual motion vectors, termed "local" motion 
vectors, have been derived for the blocks of the input image, the local 
motion vectors are supplied to a motion vector reducer. The motion 
vector reducer assigns to each block a set of motion vectors selected, 
in order of preference, from: the zero motion vector; the local motion 
vector for that block; the local motion vectors for blocks neighbouring 
that block in the input image; and a set of vectors known as "global" 
motion vectors. The global motion vectors are derived for each image 
by ranking all the (valid) local motion vectors for this image in order 
of decreasing frequency, and then selecting from the most common of the 



local motion vectors a number of unique motion vectors to be used as 
global motion vectors, A constraint is applied here to ensure that the 
global motion vectors differ from each other by at least a minimum 
amount. 

The sets of motion vectors for_all the blocks of the input image 
are then supplied to a motion vector selector. The purpose of the 
motion vector selector is to allocate to each picture element (pixel) 
of a block of an output image a single motion vector selected from the 
set of motion vectors supplied for a corresponding block of the input 
image. As this motion vector will be used in interpolation of the 
output pixel, the set of motion vectors supplied to the motion vector 
selector are tested, by performing a correlation test on blocks of 
pixels pointed to by each of the set of motion vectors, so that the 
"best" motion vector is selected for each pixel from the set of motion 
vectors for the corresponding block. The selected motion vector for 
each pixel is then supplied to a motion compensated interpolator which 
uses these motion vectors to interpolate the pixels of an output image 
from the pair of input images in dependence upon the temporal offset of 
the output image from the pair of input images. 

The purpose of the motion vector reducer is therefore to provide 
the motion vector selector with a reasonable selection of motion 
vectors for each block from which to select the most appropriate motion 
vector for each output pixel corresponding to that block. The global 
motion vectors each represent a motion which was frequently detected in 
the image as a whole, so that it is often reasonable to include these 
vectors in the sets of vectors from which the most appropriate vectors 
are selected by the motion vector selector. 

It has been observed that rapidly moving portions of a video 
image have a lower spatial detail than stationary or slowly moving 
portions. This means that even if a motion vector can be estimated 
which accurately represents the rapid motion of the object, the 
correlation test performed during vector selection (which relies on 
detecting correlation between parts of the same object in two 
successive input fields) can be unreliable. 

This invention provides a motion compensated video signal 
processing apparatus comprising: 

means for generating a plurality of motion vectors to represent 



image motion between a pair of input images of an input video signal, 
for use in interpolation of ian output image of an output video signal; 

means for detecting whether each motion vector is greater than a 
predetermined size; and 

means for restricting the use, in interpolation of the output 
image, of a motion vector detected to be greater than the predetermined 
size. 

Accordingly, even though large magnitude (long) motion vectors 
(i.e. vectors representing rapid image motion) can be generated, these 
large vectors are deliberately restricted in their use in interpolation 
of the output image. This is because the lack of spatial detail in 
fast-moving objects (e.g. due to camera integration blur) means that: 

a) the vector selection process applied to rapidly moving 
objects can be unreliable, and 

b) for rapidly moving objects, the eye is less able to detect 
the judder introduced by non motion compensated interpolation; this 
judder can therefore be less subjectively disturbing than motion 
compensated interpolation using erroneously selected vectors. 

In one preferred embodiment each motion vector comprises a 
plurality of coordinate values representing image motion along 
respective coordinate axes; and the means for detecting comprises means 
for comparing each of the coordinate values with a respective 
predetermined threshold value associated with each of the coordinate 
axes. 

In another preferred embodiment each motion vector again 
comprises a plurality of coordinate values representing image motion 
along respective coordinate axes; and the means for detecting comprises 
means for generating a magnitude value from the coordinate values for 
each motion vector, the magnitude value representing the magnitude of 
the image motion represented by that motion vector, and means for 
comparing the magnitude value with a predetermined threshold value. In 
this case, it is preferred that the means for generating a magnitude 
value is operable to detect the square root of a weighted sum of the 
squares of the coordinate values. A weighted sum may be used to 
reflect a different scaling of motion vector coordinate values in the 
various coordinate directions. Alternatively, the weighting could be 
applied equally to the individual squared coordinate values. 



Although the coordinate axes could lie along various directions 
with respect to the input images, it is preferred that the coordinate 
axes represent vertical and horizontal image motion. 

Preferably each motion vector represents image motion of a 
respective block of one input image of the pair between that image and 
the other image of the pair, and the apparatus comprises: means for 
deriving a set of global motion vectors comprising a plurality of 
distinct motion vectors selected from the most common of the plurality 
of motion vectors; and means for assigning to each of the blocks a 
group of motion vectors selected from the zero motion vector, the 
motion vector representing image motion of that block and the global 
motion vectors. 

In order to prevent global vectors being selected which only 
represent the motion of a small part of the image, it is preferred that 
each motion vector selected as a global vector has at least a 
predetermined number of occurrences among the plurality of motion 
vectors. 

In one embodiment the "long" motion vectors can be prevented from 
being used as a global motion vector. To this end, it is preferred 
that the means for restricting comprises means for preventing selection 
as a global motion vector of a motion vector detected to be greater 
than the predetermined size. 

In another embodiment, the use of the "long" motion vector can be 
further restricted, by employing a means for restricting comprising 
means, responsive to a detection that the motion vector representing 
motion of a block is greater than the predetermined size, for 
controlling the means for assigning to assign to that block only the 
zero motion vector and the motion vector representing motion of that 
block . 

In a further preferred embodiment, the "long" motion vectors can 
be completely discarded, by employing a means for restricting 
comprising means for preventing the use, in interpolation of an output 
image, of a motion vector detected to be greater than the predetermined 
size- 

In order that different measures can be taken for motion vectors 
in two different size ranges, it is preferred that the apparatus 
comprises: means for detecting whether each motion vector is greater 



than a second predetermined size, the second predetermined size being 
greater than the f irstmentioned predetermined size; and means for 
preventing the use, in interpolation of the output image, of a motion 
vector detected to be greater than the second predetermined size* 

In another preferred embodiment the means for restricting 
comprises means for setting a flag, associated with each motion vector, 
to indicate whether that motion vector is greater than the 
predetermined size, and the apparatus comprises: means for selecting a 
motion vector, from the plurality of motion vectors, for use in 
interpolation of each pixel of the output image; a motion compensated 
pixel interpolator operable according to at least two modes of 
operation; and means for selecting one of the modes of operation of the 
motion compensated pixel interpolator, for interpolation of a pixel of 
the output image, in dependence on whether the flag, corresponding to 
the motion vector selected for use in interpolation of that pixel, is 
set. In this way, the interpolation operation can be matched to the 
motion characteristics of the video signal. 

Although various arrangements of the pixel interpolator can be 
employed to give different interpolation operations in dependence on 
whether the flag is set, it is preferred that the pixel interpolator is 
operable, according to one of the modes of operation, to interpolate 
between blocks of each of the two images, pointed to by the selected 
motion vector, according to a combining ratio dependent upon the 
temporal position of the output image with respect to the pair of input 
images; and the pixel interpolator is operable, according to another 
mode of operation, to interpolate between blocks of each of the two 
images, pointed to by the zero motion vector, according to an equal 
combining ratio. 

As mentioned above, a problem can arise when the global vectors 
are derived from a relatively small number of occurrences. For 
example, if the majority of the image is stationary, except for a small 
moving object, the majority of the motion vectors for that image would 
be close to zero. The vectors representing the stationary parts of the 
image may differ slightly due to noise or aliasing, but would still be 
too close to one another to be selected individually as global vectors. 
The next most commonly occurring vector, which would therefore be 
selected as a global vector, would be the vector derived from the small 



moving object. If this vector is passed to vector selection as one of 
the four vectors associated with a non-moving block of the image, it 
may be spuriously selected during vector selection. The use of a 
grossly incorrect vector in this way can result in visible artifacts 
being generated. 

Viewed from a second aspect this invention provides a motion 
compensated video signal processing apparatus in which motion vectors 
are generated to represent image motion between a pair of input images 
of an input video signal, the apparatus comprising: 

means for generating a plurality of local motion vectors to 
represent image motion of respective blocks of one input image of the 
pair between that image and the other image of the pair; 

means for deriving from the local motion vectors a set of global 
motion vectors comprising a plurality of distinct motion vectors 
selected from the most common of the local motion vectors, each global 
vector having at least a predetermined number of occurrences among the 
local motion vectors; and 

means for assigning to each of the blocks a group of motion 
vectors selected from the zero motion vector, the local motion vector 
for that block and the global motion vectors. 

This aspect of the invention addresses the problem of using 
global motion vectors derived from a relatively small number of 
occurrences, by preventing the selection as a global motion vector of 
a local vector which occurs fewer than a predetermined number of times. 

Preferably the means for deriving comprises means for preventing 
the selection, as global motion vectors, of two or more local motion 
vectors which differ from one another by less than a predetermined 
amount . 

In a preferred embodiment, the predetermined number of 
occurrences can be adjusted under the control of an operator. 

In an advantageously simple embodiment the apparatus comprises 
means for storing a frequency array comprising a plurality of 
addressable memory locations, each possible value of the motion vectors 
representing the address of a respective memory location; means for 
incrementing a frequency value stored at the memory location 
corresponding to each of the plurality of motion vectors; and means for 
detecting those memory locations having the highest stored frequency 
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values. 

Apparatus according to the invention is particularly usefully 
employed in a television standards conversion apparatus. 

Viewed from a third aspect this invention provides a method of 
motion compensated video signal processing, the method comprising the 
steps of: 

generating a plurality of motion vectors to represent image 
motion between a pair of input images of an input video signal, for use 
in interpolation of an output image of an output video signal; 

detecting whether each motion vector is greater than a 
predetermined size; and 

restricting the use, in interpolation of the output image, of a 
motion vector detected to be greater than the predetermined size. 

Viewed from a fourth aspect this invention provides a method of 
motion compensated video signal processing in which motion vectors are 
generated to represent image motion between a pair of input images of 
an input video signal, the method comprising the steps of: 

generating a plurality of local motion vectors to represent image 
motion of respective blocks of one input image of the pair between that 
image and the other image of the pair; 

deriving from the local motion vectors a set of global motion 
vectors comprising a plurality of distinct motion vectors selected from 
the most common of the local motion vectors, each global vector having 
at least a predetermined number of occurrences among the local motion 
vectors; and 

assigning to each of the blocks a group of motion vectors 
selected from the zero motion vector, the local motion vector for that 
block and the global motion vectors. 

The invention will now be described by way of example with 
reference to the accompanying drawings, throughout which like parts are 
referred to by like references, and in which: 

Figure 1 is a schematic block diagram of a motion compensated 
television standards conversion apparatus; 

Figure 2 is a schematic block diagram of a motion vector reducer; 

Figure 3 is a schematic block diagram of a global vector 
detector; and 

Figures 4 to 7 show modifications to the apparatus of Figure 1 to 
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control the use of motion vectors in response to a long vector flag. 

Figure 1 is a schematic block diagram of a motion compensated 
television standards conversion apparatus. The apparatus receives an 
input interlaced digital video signal 50 (e.g. an 1125/60 2:1 high 
5 definition video signal (HDVS)) and generates an output interlaced 

digital video signal 60 (e.g a 1250/50 2:1 signal). 

The input video signal 50 is first supplied to an input 
buffer/packer 110. In the case of a conventional definition input 
signal, the input buffer/packer 110 formats the image data into a high 

10 definition (16:9 aspect ratio) format, padding with black pixels where 

necessary. For a HDVS input the input buffer/packer 110 merely 
provides buffering of the data. 

The data are passed from the input buffer/packer 110 to a matrix 
circuit 120 in which (if necessary) the input video signal's 

15 colorimetry is converted to the standard "CCIR recommendation 601" 

(Y,Cr,Cb) colorimetry. 

From the matrix circuit 120 the input video signal is passed to 
a time base changer and delay 130, and via a sub-sampler 170 to a 
subsampled time base changer and delay 180. The time base changer and 

20 delay 130 determines the temporal position of each field of the output 

video signal, and selects the two fields of the input video signal 
which are temporally closest to that output field for use in 
interpolating that output field. For each field of the output video 
signal, the two input fields selected by the time base changer are 

25 appropriately delayed before being supplied to an interpolator lUO in 

which that output field is interpolated- A control signal t, 
indicating the temporal position of each output field with respect to 
the two selected input fields, is supplied from the time base changer 
and delay 130 to the interpolator 1*40. 

30 The subsampled time base changer and delay 180 operates in a 

similar manner, but using spatially subsampled video supplied by the 
subsampler 170. Pairs of fields, corresponding to the pairs selected 
by the time base changer 130, are selected by the subsampled time base 
changer and delay 180 from the subsampled video, to be used in the 

35 generation of motion vectors. 

The time base changers 130 and 180 can operate according to 
synchronisation signals associated with the input video signal, the 



output video signal, or both. In the case in which only one 
synchronisation signal is supplied, the timing of fields of the other 
of the two video signals is generated deterniinis tically within the time 
base changers 130, 180. 

The pairs of fields of the subsampled input video signal selected 
by the subsampled time base changer* and delay 180 are supplied to a 
motion processor 185 comprising a direct block matcher 190, a 
correlation surface processor 200, a motion vector estimator 210, a 
motion vector reducer 220, a motion vector selector 230 and a motion 
vector post-processor 240. The pairs of input fields are supplied 
first to the direct block matcher 190 which calculates correlation 
surfaces representing the spatial correlation between search blocks in 
the temporally earlier of the two selected input fields and (larger) 
search areas in the temporally later of the two input fields. 

From the correlation surfaces output by the block matcher 190 f 
the correlation surface processor 200 generates a larger number of 
interpolated correlation surfaces, which are then passed to the motion 
vector estimator 210. The motion vector estimator 210 detects points 
of greatest correlation in the interpolated correlation surfaces. (The 
original correlation surfaces actually represent the difference between 
blocks of the two input fields; this means that the points of maximum 
correlation are in fact minima on the correlation surfaces, and are 
referred to as "minima"). In order to detect a minimum, additional 
points on the correlation surfaces are interpolated, providing a degree 
of compensation for the loss of resolution caused by the use of 
subsampled video to generate the surfaces. From the detected minimum 
on each correlation surface, the motion vector estimator 210 generates 
a motion vector which is supplied to the motion vector reducer 220* 

The motion vector estimator 210 also performs a confidence test 
on each generated motion vector to establish whether that motion vector 
is significant above the average data level, and associates a 
confidence flag with each motion vector indicative of the result of the 
confidence test. The confidence test, known as the "threshold" test, 
is described (along with certain other features of the apparatus of 
Figure 1) in GB-A-2 231 7^9 and British Patent Application number 
9307^2.5. 

A test is also performed by the motion vector estimator 210 to 
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detect whether each vector is aliased. In this test, the correlation 
surface (apart from an exclusion zone around the detected minimum) is 
examined to detect the next lowest minimum. If this second minimum 
does not lie at the edge of the exclusion zone, the motion vector 
5 derived from the original minimum is flagged as being potentially 

aliased. 

The motion vector reducer 220 operates to reduce the choice of 
possible motion vectors for each pixel of the output field, before the 
motion vectors are supplied to the motion vector selector 230. The 

10 output field is notionally divided into blocks of pixels, each block 

having a corresponding position in the output field to that of a search 
block in the earlier of the selected input fields. The motion vector 
reducer compiles a group of four motion vectors to be associated with 
each block of the output field, with each pixel in that block 

15 eventually being interpolated using a selected one of that group of 

four motion vectors. 

Vectors which have been flagged as "aliased" are re-qualified 
during vector reduction if they are identical, or similar within a 
selectable threshold, to non- flagged vectors in nearby blocks. 

20 As part of its function, the motion vector reducer 220 counts the 

frequencies of occurrence of "good" motion vectors (i.e. motion vectors 
which pass the confidence test and the alias test, or which were re- 
qualified as non-aliased), with no account taken of the position of the 
blocks of the input fields used to obtain those motion vectors. The 

25 good motion vectors are then ranked in order of decreasing frequency. 

The most common of the good motion vectors which are significantly 
different to one another are then classed as "global" motion vectors. 
Three motion vectors which pass the confidence test are then selected 
for each block of output pixels and are supplied, with the zero motion 

30 vector, to the motion vector selector 230 for further processing. 

These three selected motion vectors are selected in a predetermined 
order of preference from: 

(i) the motion vector generated from the corresponding search 
block (the "local" motion vector"); 

35 (ii) those generated from surrounding search blocks 

("neighbouring" motion vectors); and 

(iii) the global motion vectors- 



The motion vector selector 230 also receives as inputs the two 
input fields which were selected by the subsampled time base changer 
and delay 180 and which were used to calculate the motion vectors, 
along with the control signal t. These fields are suitably delayed so 
that they are supplied to the motion vector selector 230 at the same 
time as the vectors derived from them. The motion vector selector 230 
supplies an output comprising one motion vector per pixel of the output 
field. This motion vector is selected from the four motion vectors for 
that block supplied by the motion vector reducer 220. 

The vector selection process involves detecting the degree of 
correlation between test blocks of the two input fields pointed to by 
a motion vector under test. The motion vector having the greatest 
degree of correlation between the test blocks is selected for use in 
interpolation of the output pixel. A "motion flag" is also generated 
by the vector selector. This flag is set to "static" (no motion) if 
the degree of correlation between blocks pointed to by the zero motion 
vector is greater than a preset threshold. 

The vector post-processor reformats the motion vectors selected 
by the motion vector selector 230 to reflect any vertical or horizontal 
scaling of the picture, and supplies the reformatted vectors to the 
interpolator 1*10. Using the motion vectors, the interpolator 1*10 
interpolates an output field from the corresponding two (non- 
subsampled) interlaced input fields selected by the time base changer 
and delay 130, taking into account any image motion indicated by the 
motion vectors currently supplied to the interpolator 1^0. 

If the motion flag indicates that the current output pixel lies 
in a moving part of the image, pixels from the two selected fields 
supplied to the interpolator are combined in relative proportions 
depending on the temporal position of the output field with respect to 
the two input fields (as indicated by the control signal t) . so that a 
larger proportion of the nearer input field is used. If the motion 
flag is set to "static" then the temporal weighting is fixed at 50% of 
each input field. The output of the interpolator 140 is passed to an 
output buffer 150 for output as a high definition output signal, and to 
a down-converter l60 which generates a conventional definition output 
signal 165, using the motion flag. 

The down-converter 160 allows a representation of the output of 
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the apparatus (which may be, for example, a high definition video 
signal) to be monitored, transmitted and/or recorded using conventional 
definition apparatus. This has benefits because conventional 
definition recording equipment is significantly cheaper and very much 
more widespread than high definition equipment. For example, a 
simultaneous output of conventional and high definition video may be 
required for respective transmission by terrestrial and satellite 
channels. Furthermore, if the output video signal is recorded directly 
onto film using, for example, an electron beam recorder, the down- 
converter allows a simultaneous recording to be made onto video tape. 

The subsampler 170 performs horizontal and vertical spatial 
subsarapling of the input video fields received from the matrix 120, 
before those input fields are supplied to the time base changer 180. 
Horizontal subsampling is a straightforward operation in that the input 
fields are first prefiltered by a half-bandwidth low pass filter (in 
the present case of 2:1 horizontal decimation) and alternate video 
samples along each video line are then discarded, thereby reducing by 
one half the number of samples along each video line. 

Vertical subsampling of the input fields is complicated by the 
fact that the input video signal 50 is interlaced. This means that 
successive lines of video samples in each interlaced field are 
effectively two video lines apart, and that the lines in each field are 
vertically displaced from those in the preceding or following field by 
one video line of the complete frame. 

The construction and operation of the motion vector reducer 220 
will now be described in more detail. 

A block diagram of the motion vector reducer 220 is shown in 
Figure 2. The motion vectors, and their associated flags, derived by 
the motion vector estimator 210 are supplied via an input 300 of the 
motion vector reducer 220 to a ridge minima requalifier 310. The ridge 
minima requalifier 310 "repairs" motion vectors derived from elongate 
("ridge") minima, by detecting whether another motion vector along the 
ridge direction has a matching coordinate at 90° to the ridge direction; 
if so, then that coordinate is assigned to the ridge vector and a 
"ridge flag" associated with that vector is reset. The ridge minima 
requalifier then supplies the vectors, and associated flags, to an 
alias vector validator 320 and a global vector detector 330. The alias 
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vector validator 320 performs a test to check whether vectors indicated 
as potentially aliased can be justifiably requalified as non-aliased, 
and the vectors, and resulting associated flags, form one input to a 
global vector res trie ter 3^0. The global vector detector 330 derives 
a set of global motion vectors from the vectors supplied thereto. The 
global vectors output by the global vector detector 330 form a second 
input to the global vector restricter 3^0. 

The global vector restricter 3^0 generates, for each global 
vector supplied thereto, a global mask array indicating an association 
between the global vector and various blocks of the input image to 
which the local vectors supplied from the alias vector validator 320 
correspond. The global mask arrays generated by the global vector 
restricter 3^0 are supplied to a block vector allocator 350 which also 
receives the (suitably delayed) local vectors, and associated flags, 
from the alias vector validator 320 and the global vectors from the 
global vector detector 330. The block vector allocator 350 assigns to 
each block of the input image to which the local vectors correspond a 
plurality of unique vectors which are then supplied via an output 360 
of the motion vector reducer 220 to the motion vector selector 230. 

The ridge minima requalifier 310, alias vector validator 320, 
global vector detector 330, global vector restricter 3*40 and block 
vector allocator 350 operate under the control (indicated schematically 
by dotted lines in the figure) of a microprocessor 370. The 
microprocessor 370 is also connected to the global vector detector 330 
to provide for optional manual intervention in the global vector 
detection process to adjust various parameters. The adjusted 
parameters can be entered by an operator and supplied by the 
microprocessor 370 to the global vector detector 330. A two-way 
communication link 380 is also provided to connect the microprocessor 
370 to a further, similar microprocessor (not shown). This is because 
the entire motion processor (indicated generally as 185 in Figure 1) 
may in fact be duplicated, the two motion processors performing the 
processing for alternate output fields. Thus, the vector reducer shown 
in Figure 2 may be duplicated, and the link 380 is provided for 
communication of the two microprocessors controlling the vector 
reduction operation. 

The construction and operation of certain components of the 



motion vector reducer 220 will now be described in more detail. 

The vectors, and flags, output by the ridge minima requalifier 
310 are supplied to the global vector detector 330. The purpose of the 
global vector detector is to derive from the vectors supplied thereto 
5 a set of global motion vectors comprising a plurality of unique motion 

vectors selected from the most common of the local motion vectors. In 
general terms, the global vector detector 330 ranks local motion 
vectors corresponding to a given input image in order of decreasing 
frequency, and selects the eight (in this example) most common vectors 

10 as global vectors. However, it can be advantageous to apply various 

constraints to the process for "selecting global vectors from the set of 
local vectors. In particular, even though large magnitude (long) 
motion vectors (ie vectors representing rapid image motion) can be 
generated, it is desirable to prevent such long vectors from 

15 contributing to the global vectors. This is because the lack of 

spatial detail in fast-moving objects (eg due to camera integration 
blur) means that the operation of the motion vector selector 230 
(Figure 1) for rapidly moving objects can be unreliable. It has been 
observed that rapidly moving portions of a video image have lower 

20 spatial detail than stationary or slowly moving portions. This means 

that even if a motion vector can be estimated which accurately 
represents the rapid motion of the object, the correlation test 
performed in the vector selector 230, which relies on detecting 
correlation between parts of the same object in two successive input 

25 fields, can be unreliable. It is therefore preferable to prevent such 

long vectors from being selected as global vectors to ensure that a 
large magnitude global vector cannot be incorrectly selected for 
interpolation of image areas where there is in fact only very small 
motion. This could produce subjectively disturbing effects in the 

30 output image. 

A further constraint which is applied to the global vector 
detection process is to prevent a vector being accepted as a global 
vector unless it occurs at least a predetermined number of times over 
the input image as a whole. If a local motion vector is not 

35 sufficiently common, and thus represents motion of only a very small 

part of the image, it is preferable to eliminate the possibility of 
this vector being assigned to other parts of the image during the 
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global vector detection stage. 

A schematic block diagram of a global vector detector 330 in 
which the above constraints are implemented is shown in Figure 3. The 
input vectors, and associated flags, are supplied to an amplitude 
detector 400 which compares the amplitude, ie magnitude, of each input 
vector with a predetermined magnitude threshold 4 10 to decide whether 
the vector qualifies as a "long" vector. The magnitude threshold 410 
may comprise a threshold for both the horizontal and vertical 
components of a motion vector. For example, the threshold may be set 
at half the maximum possible range, horizontally and vertically, of a 
motion vector. In this example, the maximum horizontal range of a 
motion vector is ±64 pixels, and the maximum vertical range ±32 pixels. 
Thus, the magnitude threshold 4 10 may be set at ±32 pixels horizontally 
and ±16 pixels vertically. If the magnitude of an input vector exceeds 
the magnitude threshold 410, the amplitude detector 400 sets a "long 
vector" flag (LV) . 

As an alternative, the threshold 410 could represent the maximum 
value of the vector magnitude /(x J + y-) f or a weighted sum to reflect 
a different scaling of the horizontal and vertical coordinates (e.g. 
/(4x 2 ♦ y 2 } or /(*- + 4y")). 

The input vectors supplied to the amplitude detector 400 are also 
supplied to a frequency array 420. Each entry in the frequency array 
420 is uniquely addressed by a possible value of the local motion 
vectors. In this way, a local motion vector generated from the current 
pair of input fields is used to address the array and to increment the 
array entry corresponding to the value of the motion vector. However, 
the long vector flags LV derived by the amplitude detector 400 are also 
supplied to the frequency array 420. If the long vector flag LV for a 
motion vector supplied to the frequency array 420 is set, then this 
prevents the associated motion vector from incrementing the frequency 
array . 

After all the vectors (apart from long vectors) have been counted 
in this way by the frequency array 420, an array scanner 430 scans the 
frequency array 420 and identifies the eight array entries which show 
the highest count, for which the motion vectors differ from each other 
by at least a predetermined difference threshold 435. The addresses of 
these array entries represent the values of the eight most common local 



vectors (apart from long vectors) for the corresponding pair of input 
images. The value of the count held by the frequency array for each of 
these eight vectors (that is, the frequency of occurrence of the 
vectors) is supplied by the array scanner 430 to a comparator 440 at 
the same time as the corresponding vector is supplied to an input of a 
switch 450. The comparator 440 compares the count, or frequency of 
occurrence, corresponding to each vector with a predetermined threshold 
number or "global threshold" , and provided the count exceeds the global 
threshold, the corresponding vector will be supplied to the output 460 
of the switch 450 as a global motion vector. (A suitable value for the 
global threshold may vary depending on the particular image material 
being processed. By way of example, however, the global threshold may 
be set at 0.5% of the total number of local motion vectors derived from 
an input image.) If the count corresponding to any of the eight most 
common vectors does not exceed the global threshold, indicating that 
the vector does not occur sufficiently frequently to qualify as a 
global vector, then the comparator 440 generates an output to control 
the switch 450 such that the corresponding vector is not passed to its 
output 460 as a global vector, but is replaced by the zero motion 
vector. Assuming, however, that all eight of the most common vectors 
pass the global threshold test, then a set of eight unique global 
motion vectors will be supplied to the output 460 of the switch 450. 

It will be appreciated that various further constraints could be 
applied in the global vector detector 330. In particular, any input 
local motion vectors for which the corresponding ridge or alias flags 
are set, or for which the confidence flag indicates that the vector 
failed the confidence test, may be prevented from contributing to the 
global vectors in a similar manner to long vectors, ie by preventing 
such vectors from incrementing the frequency array 420. 

As indicated in Figure 3, the long vector flag LV derived for 
each of the input local motion vectors is supplied by the global vector 
detector 330, after an appropriate delay, to the block vector allocator 
350 (Figure 2) for use in the subsequent vector reduction processing of 
the local vectors. 

As shown in Figure 2, the global vector detector 330 is connected 
for communication with the microprocessor 370. This allows for the 
various thresholds used 'during global vector detection to be modified, 



if required, through manual intervention by an operator. The operator 
can enter new parameters, for example via a system controller (not 
shown) , which are then supplied via the microprocessor 370 to the 
global vector detector 330. 

The global vectors output by the global vector detector 330 are 
supplied to one input of the global vector restricter 3^0. At its 
other input, the global vector restricter 3^0 receives the processed 
local motion vectors and associated flags from the alias vector 
validator 320. The purpose of the global vector x-estricter 3*40 is to 
associate each of the global vectors with blocks of the input image at 
or near areas where motion identical or similar to that represented by 
the global motion vector was originally derived. This association will 
then be used by the block vector allocator 350 to prevent inappropriate 
global vectors being assigned to blocks, that is, to prevent a block 
from one area of the input image being assigned a global vector 
representing significantly different motion of a block in a remote part 
of the image. 

The block vector allocator receives the LV flag from the 
amplitude detector 400, and can respond in a number of ways. In the 
embodiment described above, long vectors are already prevented from 
being used as global vectors; however, the block vector allocator 350 
could in fact use the flag to prevent the use of long vectors at all. 
Such an arrangement is illustrated schematically in Figure 4, in which 
the LV flag associated with each motion vector controls a multiplexer 
600 to pass either the associated vector or the zero vector to the 
input of the block vector allocator 350- 

In another arrangement, illustrated schematically in Figure 5. 
the vector selection process for a particular block is restricted to 
the possible use of the long local vector for that block or the zero 
motion vector only. As shown in Figure 5. the LV flag for each block 
controls a multiplexer 610 to pass (to vector selection) either the 
three motion vectors assigned to that block during vector reduction or 
the local motion vector plus two zero motion vectors. 

In another embodiment illustrated in Figure 6 f a further 
amplitude detector 620, similar to the amplitude detector 400, is 
employed to compare each vector with a second, higher threshold value 
630. Although the use of long vectors may in general be allowed 
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(although restricted as described above), the use of vectors which are 
greater in size than this second threshold is not allowed. This is 
achieved by the use of a multiplexer 640 which passes each vector to 
the block vector allocator 350 only if that vector does not exceed the 
second threshold size 630. 

In a further embodiment, the LV flag can be used to control the 
operation of the interpolator 140. This arrangement is illustrated in 
Figure 7, in which the LY flag controls a multiplexer 650, causing it 
to pass either the control signal t or a fixed value of 0.5 to the 
interpolator 140. This means that the use of temporal weighting 
(described above) can be inhibited when the LV flag is set. This can 
reduce disturbing image artifacts such as the visible judder caused by 
beating effects that may be observed on moving portions of "shuttered" 
images . 
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1. Motion compensated video signal processing apparatus comprising: 
means for generating a plurality of motion vectors to represent 

image motion between a pair of input images of an input video signal, 
for use in interpolation of an output image of an output video signal; 

means for detecting whether each motion vector is greater than a 
predetermined size; and 

means for restricting the use, in interpolation of the output 
image, of a motion vector detected to be greater than the predetermined 
size. 

2. Apparatus according to claim 1, in which: 

each motion vector comprises a plurality of coordinate values 
representing image motion along respective coordinate axes; and 

the means for detecting comprises means for comparing each of the 
coordinate values with a respective predetermined threshold value 
associated with each of the coordinate axes. 

3- Apparatus according to claim 1, in which: 

each motion vector comprises a plurality of coordinate values 
representing image motion along respective coordinate axes; and 

the means for detecting comprises means for generating a 
magnitude value from the coordinate values for each motion vector, the 
magnitude value representing the magnitude of the image motion 
represented by that motion vector, and means for comparing the 
magnitude value with a predetermined threshold value. 

4. Apparatus according to claim 3. in which the means for generating 
a magnitude value is operable to detect the square root of a weighted 
sum of the squares of the coordinate values. 

5- Apparatus according to any one of claims 2 to 4, in which the 
coordinate axes represent vertical and horizontal image motion. 

6. Apparatus according to any one of the preceding claims, in which 
each motion vector represents image motion of a respective block of one 
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input image of the pair between that image and the other image of the 
pair, the apparatus comprising: 

means for deriving* a set of global motion vectors comprising a 
plurality of distinct motion vectors selected from the most common of 
the plurality of motion vectors; and 

means for assigning to each of the blocks a group of motion 
vectors selected from the zero motion vector, the motion vector 
representing image motion of that block and the global motion vectors. 

7. Apparatus according to claim 6, in which each motion vector 
selected as a global vector has at least a predetermined number of 
occurrences among the plurality of motion vectors. 

8. Apparatus according to claim 6 or claim 7. in which the means for 
restricting comprises means for preventing selection as a global motion 
vector of a motion vector detected to be greater than the predetermined 
size. 

9. Apparatus according to any one of claims 6 to 8, in which the 
means for restricting comprises means, responsive to a detection that 
the motion vector representing motion of a block is greater than the 
predetermined size, for controlling the means for assigning to assign 
to that block only the zero motion vector and the motion vector 
representing motion of that block. 

10. Apparatus according to any one of claims 1 to 5, in which the 
means for restricting comprises means for preventing the use, in 
interpolation of an output image, of a motion vector detected to be 
greater than the predetermined size. 

.11. Apparatus according to any one of claims 1 to 5, comprising: 

means for detecting whether each motion vector is greater than a 

second predetermined size, the second predetermined size being greater 

than the firs tmentioned predetermined size; and 

means for preventing the use, in interpolation of the output 

image, of a motion vector detected to be greater than the second 

predetermined size. 



12. Apparatus according to any one of claims 1 to 5. in which the 
means for restricting comprises means for setting a flag, associated 
with each motion vector, to indicate whether that motion vector is 
greater than the predetermined size, the apparatus comprising: 

means for selecting a motion vector, from the plurality of motion 
vectors, for use in interpolation of each pixel of the output image; 

a motion compensated pixel interpolator operable according to at 
least two modes of operation; and 

means for selecting one of the modes of operation of the motion 
compensated pixel interpolator, for interpolation of a pixel of the 
output image, in dependence on whether the flag, corresponding to the 
motion vector selected for use in interpolation of that pixel, is set. 

13- Apparatus according to claim 12, in which: 

the pixel interpolator is operable, according to one of the modes 
of operation, to interpolate between blocks of each of the two images, 
pointed to by the selected motion vector, according to a combining 
ratio dependent upon the temporal position of the output image with 
respect to the pair of input images; and 

the pixel interpolator is operable, according to another mode of 
operation, to interpolate between blocks of each of the two images, 
pointed to by the zero motion vector, according to an equal combining 
ratio. 

14. Motion compensated video signal processing apparatus in which 
motion vectors are generated to represent image motion between a pair 
of input images of an input video signal, the apparatus comprising: 

means for generating a plurality of local motion vectors to 
represent image motion of respective blocks of one input image of the 
pair between that image and the other image of the pair; 

means for deriving from the local motion vectors a set of global 
motion vectors comprising a plurality .of distinct motion vectors 
selected from the most common of the local motion vectors, each global 
vector having at least a predetermined number of occurrences among the 
local motion vectors; and 

means for assigning to each of the blocks a group of motion 
vectors selected from the zero motion vector, the local motion vector 



for that block and the global motion vectors. 

15. Apparatus according to claim 14, in which the means for deriving 
comprises means for preventing the selection, as global motion vectors, 
of two or more local motion vectors which differ from one another by 
less than a predetermined amount. 

16. Apparatus according to claim 1*1 or claim 15 . in which the 
predetermined number of occurrences can be adjusted under the control 
of an operator. 

17. Apparatus according to any one of claims 14 to 16, comprising: 
means for storing a frequency array comprising a plurality of 

addressable memory locations, each possible value of the motion vectors 
representing the address of a respective memory location; 

means for incrementing a frequency value stored at the memory 
location corresponding to each of the plurality of motion vectors; and 

means for detecting those memory locations having the highest 
stored frequency values. 

18. Television standards conversion apparatus comprising apparatus 
according to any one of the preceding claims. 

19. A method of motion compensated video signal processing, the 
method comprising the steps of : 

generating a plurality of motion vectors to represent image 
motion between a pair of input images of an input video signal, for use 
in interpolation of an output image of an output video signal; 

detecting whether each motion vector is greater than a 
predetermined size; and 

restricting the use, in interpolation of the output image, of a 
motion vector detected to be greater than the predetermined size. 

20. A method of motion compensated video signal processing in which 
motion vectors are generated to represent image motion between a pair 
of input images of an input video signal, the method comprising the 
steps of: 
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generating* a plurality of local motion vectors to represent image 
motion of respective blocks of one input image of the pair between that 
image and the other image of the pair; 

deriving from the local motion vectors a set of global motion 
vectors comprising a plurality of distinct motion vectors selected from 
the most common of the local motion vectors, each global vector having 
at least a predetermined number of occurrences among the local motion 
vectors ; and 

assigning to each of the blocks a group of motion vectors 
selected from the zero motion vector, the local motion vector for that 
block and the global motion vectors. 

21. Motion compensated video signal processing apparatus 
substantially as hereinbefore described with reference to the 
accompanying drawings . 

22. A method of motion compensated video signal processing, the 
method being substantially as hereinbefore described with reference to 
the accompanying drawings. 

23- Television standards conversion apparatus substantially as 
hereinbefore described with reference to the accompanying drawings. 
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